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Sir: 



We, DARRYL E. RUBIN, ANDREW C. BAIRD, JOHN L. BEEZER, JONATHAN C 
CLUTS, and SUSAN D. WOOLF, hereby declare 1 that: 



1) We are named as joint inventors of the above-captioned application, U.S. Application 
Serial No. 09/456,249, and all claims presently pending therein; 

2) I, DARRYL E. RUBIN, am presently employed by Microsoft Corporation 
(Microsoft) and have been since June 1986. Microsoft is the assignee of the above- 
identified application. 



1 Each numbered declaration is a joint declaration unless an individual reference has been made. In such a case, the 
referenced individual is making the numbered declaration. 
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3) I, JONATHAN C CLUTS, am presently employed by Microsoft and have been since 
October 1990. Microsoft is the assignee of the above-identified application. 

4) I, SUSAN D. WOOLF, am presently employed by Microsoft and have been since 
October 1995. Microsoft is the assignee of the above-identified application. 

5) We, ANDREW C. BAIRD and JOHN L. BEEZER, were formerly employed by 
Microsoft. 

6) We were employed by Microsoft during development of the above-identified 
invention. 

7) Prior to May 8, 1999, the earliest priority date of Bizweb2000.com - screenshots of 
an e-book, published 5/8/1999, pages 1-4 ("Bizweb2000 reference"), we conceived of 
and reduced to practice the invention recited in the claims of the above-captioned 
application. 

8) Conception and actual reduction to practice occurred prior to May 8, 1999, as 
evidenced by pages 1, 2, 4, 5, and 7 of the Patent Predisclosure Document attached as 
Exhibit A. 

9) Support for claims 1-5 of the above-captioned application can be found, among other 
places, at least within Exhibit A prepared prior to May 8, 1999. 

10) Support for claims 1-5 can be found at least within Exhibit A, among other places, in 
the bullet points on page 1, and the descriptions at the bottom of page 1 and the top of 
page 2. In addition, the screenshots on pages 5 and 7 support claims 1-5. Claim 2 is 
additionally supported, among other places, by the description on page 4 of Exhibit 
A. 

11) Attached Exhibit A has not been altered since it was originally prepared, except for 
the redaction of dates and the removal of pages 3, 6, and 8, which are not pertinent to 
claims 1-5. 

12) Each of us individually represents that we are over 18 years of age and of competent 
mind. 
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13) All statements made of our own knowledge are true and all statements made on 
information and belief are believed to be true; and further, these statements were 
made with the knowledge that willful, false statement so made are punishable by fine 
or imprisonment or both, under 18 U.S.C § 1001 and that such willful, false 
statements may jeopardize the validity of the above-identified application or any 
patent issuing thereon. 
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Darryl E. Rubin, Date 
Microsoft Corporation 
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Microsoft Patent Predisclosure Document 



Title of Invention: Electronic Web-based Interface for Reading and Annotation 
Date: 

Document Author(s): Darryl Rubin, Jonathan Chits, Suze Woolf 



Introduction 

[Please provide a high level description of the invention, including the names of the people who contributed to the invention.] 

An architecture for a window-less, scroll-less, desktop-less and save-less HTML-based universal viewer 
interface which allows the user to read, annotate, collaborate and perform other tasks typical of knowledge 
work, as well as alter the interface to best suit their work patterns. 

Motivation for the Invention: 

[Describe (I) the problem addressed by the invention (e.g., limitations of prior products of Microsoft, or others), and (2) your 
solution to the problem (including what "new" things your invention does and a high-level description of how it does them) J 

• Create a system that is simpler (fewer commands to achieve comparable results) and more powerful 
(achieve results that weren't possible) than today's desktop metaphor by unifying content and user 
interface into a single, dynamic content mechanism that uses fewer kinds of user interface controls in a 
more general way. 

• Provide a new metaphor for finding, consuming, storing, sharing and organizing content of interest to 
the user because many users find the desktop metaphor confusing and difficult to learn. 

• Provide a framework for more "natural" styles of interacting, employing a minimum number common 
gestures such as touch, hold, erase, draw or write. Currently most desktop computers require a 
keyboard and mouse in order to interact with them; most pen-enabled palmtop computers have 
cumbersome means of interaction, 

• Scales from small handheld devices to desktop computers. Currently desktop metaphors applied to 
small form factor devices are cluttered and difficult to use. 

• Reduce complexity and confusion of the multi-windowed interface by using a model of a single main 
window. Usability data show that users of multi-windowed systems don't always know which actions 
will produce results in which window. 

• Provide greater flexibility to users, content developers and third-party software developers by 
providing broader capabilities for customizing the user interface and creating dynamic content. 

• Provide a save-less model, so that users don't need to explicitly save their work. 

• Provide rich support for audio note taking with the ability to correlate and synchronize audio and 
textual material and to review and retrieve audio notes. 

• Provide automatic and transparent synchronization between a user's computers (e.g. handheld and 
desktop). Current device-to-device and device-to-PC synchronization schemes are not seamless and 
require a great deal of configuration and attention from the user. 

• Make the process of getting help about a function be the same as the process for carrying out the 
function. In today's user interfaces, methods for getting assistance with an interface or with content are 
currently separate from the content and often require completely different interactions than consuming 
the content 

• Make sharing and collaborating on documents easy and automatic. Today's devices and PCs have a 
"single-user model" at the heart of their interface metaphors; sharing content and annotations is 
difficult and non-intuitive. 

Description of the Invention: 

[Describe your proposed implementation of the invention, including the architecture and design details of the implementation. The 
design details should include a description of the component parts of, and individual operations performed by, your 
implementation. The use of a specific example, showing how the invention solves the problem being addressed, can be particularly 
helpful. You should also mention whether you have thought of any other implementations, or applications of, your invention. In 
most cases, 1-2 pages of description should be adequate to start the patent application process, although a more detailed 
description may greatly enhance the efficiency of the process.] 

Windowless Interface - The display may be divided into non-windowing regions in which different 
content may be displayed. Each region may be individually navigated without changing focus or "window" 
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state. Each region may include concurrent active html links without changing focus. The benefits of this 
architecture include reducing the complexity associated with both the design and use of the interface while 
still providing access to and interaction with multiple sources of concurrent content (See also pinning 
(ePad) 2.doc.) 

HTML based interface - The interface is implemented entirely as an HTML document, except that 
some links are links to scripts rather than to other documents or sites; activating such a link (such as by 
touching it) runs the linked-to script, thus making the link act as a command. Even the desktop metaphor of 
files and folders is expressed as lists of links on a page. 

Additionally, links have properties that govern their appearance and behavior. For example, a link's 
properties may dictate that it appear as a blue underscored text string as in today's browser's, or as a 3D 
button, as a graphic icon, as a thumbnail image of the content being linked to, or even as an embedded 
frame that is open on the content being linked to. The ability to control a link's appearance and behavioral 
properties makes possible rich authoring and customization of content and the user interface. 

The user may completely customize the interface (unless authored-in content controls prevent editing). 
Because there is no distinction between the interface and the content, the kinds of normal editing commands 
the user might use to alter content can be used to customize the interface. What's more, the many varied 
interface widgets that users currently must understand in order to perform useful work, such as dialog 
boxes, drop-down list boxes, etc can be done away with. Everything is simply a link on a page. Some of 
these links may be rendered as buttons to offer instant-gestalt affordances to new users, but architecturally 
they are links that may be used to search, view, and navigate either the interface or the content being 
viewed. Whether you are moving back/forward, sorting, or bookmarking a content page or an interface 
page, the concept and actions are exactly the same. 

The ability to review files, folders, mail, etc by navigating through a list of the links contemporaneously 
is enabled by this architecture and to a lesser extent by the Windowless UI architecture. History, notes, 
clippings and other system-provided directories have many unique attributes as a result of the HTML-based 
architecture also summarized below. 

Rich Navigation - The interface contains methods for paging within a document (previous/next) and 
for traversing chronology (history through back/forward). A rich model for navigation is enabled by 
attaching properties to links. For example, holding on a link offers additional information about where that 
link will lead (Link look-ahead might be displayed in thumbnail form and further holding might result in a 
graphical map of the links attached to that prospective destination.). Back and forward buttons by default 
display thumbnail views of the pages that tapping that button will lead to. 

History (all the pages the user has viewed) may be viewed in a number of ways: by time, by 
appearance, by site, document, section, page, etc. Since all of history is saved, the users' sequence may be 
viewed in a number of ways: as nodes with side tracking branches, as a linear list, or as a combination of 
most recently viewed pages and last few task categories. 

A key difference between the navigation mechanism of this invention and the prior art is that all 
navigation actions are saved in the navigational history. For example, if a user navigates through a set of 
links then goes "back" several steps, and then links through a different set of links, the user is still able to go 
**back" and then retrace their original path of links they navigated In today's browser's this is not 
possible— recollection of the first set of documents that were "backed" over is lost 

Annotations - The interface allows the user to interact with content and interface so as to make it more 
memorable, via bookmarks, clippings, highlights, overlaid and embedded ink and audio notes (See files: 
Book position (ePad) 2.doc, Ink layers on a browser.doc and Bookmark (ePad) 2.doc as well as additional 
documents to be created.) These annotations may be performed with not only different input methods, but 
adapted to the most convenient or natural one, such as a finger for highlighting, a stylus for writing or 
doodling, and speech for lengthier commentary. 

The annotations exist apart from the content (as files with links to the content); therefore they can be 
displayed not only layered on the content in appropriate positions within the content, but in other locations 
and visualizations. That is, each of these annotations is available to the user not only in situ, in the content 
where it was created, but cross-referenced in automatically-created indices which can be manipulated much 
as described in History above: by time, by appearance, by site, document, section, page, etc. Likewise they 
might be shared, selectively shared or kept private with the kind of functionality described under 
"Collaboration." 
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navigate their view of the shared document to other than the currently shared page, thus automatically 
disengaging from the shared display, and to reengage on the shared page at any time. The interface uses a 
Floor request/grant/relinquish model for controlling the group display. Users are able to request control of 
the current group display, grant the request and relinquish control (if currently in control). Side chats can 
be initiated with other members attending the meeting by using the "hold-for-more-information" method: 
holding on a visual representation of users in the room and then selecting the chat option from a content 
menu. Since in this interface model, presentations are just documents, users can invoke all the standard 
interface methods such as written and audio note taking, searching, bookmarking, etc. on both the public 
presentation (if they have the floor), or on their individual private copy. Just as with other annotations, these 
methods result in a file stored separately from the content, but linked to it in precisely the locations the user 
annotated. 

Help Book - Because the commands in the interface are simply links on a page, there isn't any 
difference between documentation, help, wizards and the other ways we currently attempt to help a user 
perform an action. Asking for help by clicking on a help link summons an appropriate help page composed 
on the fly from the content of the help book. The principles with underlie this composition are contextual: 
that is, if the user currently in the "list" or '"books" of Mail Messages, the help system infers that requests 
for assistance are about how to perform work with Mail Messages. 

Related Ideas addressed in additional pre-disclosure docs 

1. Using pinning to preview list of contents, e.g., email, folders, contacts, notes, etc.; navigate within a 
book; or to access different content, (file: Pinning (ePad) 2.doc). Different than Outlook previews 
because it is accomplished through html links and can be done with any list of objects. 

2. History can be manipulated, sorted, viewed in different ways, and the like. One feature of the history 
implementation is maintaining all branch extremities and including flexibility for presenting different 
arrangements for viewing the branches. 

3. Notes can also be manipulated, sorted, viewed in different ways but also can be selected resulting in not 
only opening the note but also the book where the note is linked. 

4. Unique thumbnail view UI for notes, history and other objects. 

5. Commands implemented through links 

Diagrams and Flow Charts: 

[To support the description provided above, please include: (a) at least one block diagram showing the architecture of the system 
that implements your invention, and (b) at least one diagram illustrating the primary steps performed by your invention.] 
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DECLARATION UNDER 37 C.F.R. S 1.131 



Sir: 

We, DARRYL E. RUBIN, ANDREW C. BAIRD, JOHN L. BEEZER, JONATHAN C 
CLUTS, and SUSAN D. WOOLF, hereby declare 1 that: 

1) We are named as joint inventors of the above-captioned application, U.S. Application 
Serial No. 09/456,249, and all claims presently pending therein; 

2) I, DARRYL E. RUBIN, am presently employed by Microsoft Corporation 
(Microsoft) and have been since June 1986. Microsoft is the assignee of the above- 
identified application. 



1 Each numbered declaration is a joint declaration unless an individual reference has been made. In such a case, the 
referenced individual is making the numbered declaration. 
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3) I, JONATHAN C. CLUTS, am presently employed by Microsoft and have been since 
October 1990. Microsoft is the assignee of the above-identified application. 

4) I, SUSAN D. WOOLF, am presently employed by Microsoft and have been since 
October 1995. Microsoft is the assignee of the above-identified application. 

5) We, ANDREW C BAIRD and JOHN L. BEEZER, were formerly employed by 
Microsoft. 

6) We were employed by Microsoft during development of the above-identified 
invention. 

7) Prior to May 8, 1999, the earliest priority date of Bizweb2000.com - screenshots of 
an e-book, published 5/8/1999, pages 1-4 ("Bizweb2000 reference"), we conceived of 
and reduced to practice the invention recited in the claims of the above-captioned 
application. 

8) Conception and actual reduction to practice occurred prior to May 8, 1999, as 
evidenced by pages 1, 2, 4, 5, and 7 of the Patent Predisclosure Document attached as 
Exhibit A. 

9) Support for claims 1-5 of the above-captioned application can be found, among other 
places, at least within Exhibit A prepared prior to May 8, 1 999. 

10) Support for claims 1-5 can be found at least within Exhibit A, among other places, in 
the bullet points on page 1, and the descriptions at the bottom of page 1 and the top of 
page 2. In addition, the screenshots on pages 5 and 7 support claims 1-5. Claim 2 is 
additionally supported, among other places, by the description on page 4 of Exhibit 
A. 

11) Attached Exhibit A has not been altered since it was originally prepared, except for 
the redaction of dates and the removal of pages 3, 6, and 8, which are not pertinent to 
claims 1-5. 

12) Each of us individually represents that we are over 1 8 years of age and of competent 
mind. 
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13) All statements made of our own knowledge are true and all statements made on 
information and belief are believed to be true; and further, these statements were 
made with the knowledge that willful, false statement so made are punishable by fine 
or imprisonment or both, under 18 U.S.C. § 1001 and that such willful, false 
statements may jeopardize the validity of the above-identified application or any 
patent issuing thereon. 



Respectfully submitted, 



Darryl E. Rubin, Date 
Microsoft Corporation 




Andrew C. JBaird Date 



John L. Beezer Date 



Jonathan C. Gluts Date 
Microsoft Corporation 



Susan D. Woolf, Date 
Microsoft Corporation 
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Microsoft Patent Predisclosure Document 



Title of Invention: Electronic Web-based Interface for Reading and Annotation 
Date: 

Document Author(s): Darryl Rubin, Jonathan Chits, Suze Woolf 



Introduction 

[Please provide a high level description of the invention, including the names of the people who contributed to the invention.] 

An architecture for a window-less, scroll-less, desktop-less and save-less HTML-based universal viewer 
interface which allows the user to read, annotate, collaborate and perform other tasks typical of knowledge 
work, as well as alter the interface to best suit their work patterns. 

Motivation for the Invention: 

[Describe (I) the problem addressed by the invention (e.g., limitations of prior products of Microsoft, or others), and (2) your 
solution to the problem (including what "new" things your invention does and a high-level description of how it does them).] 

• Create a system that is simpler (fewer commands to achieve comparable results) and more powerful 
(achieve results that weren't possible) than today's desktop metaphor by unifying content and user 
interface into a single, dynamic content mechanism that uses fewer kinds of user interface controls in a 
more general way. 

• Provide a new metaphor for finding, consuming, storing, sharing and organizing content of interest to 
the user because many users find the desktop metaphor confusing and difficult to learn. 

• Provide a framework for more "natural" styles of interacting, employing a minimum number common 
gestures such as touch, hold, erase, draw or write. Currently most desktop computers require a 
keyboard and mouse in order to interact with them; most pen-enabled palmtop computers have 
cumbersome means of interaction. 

• Scales from small handheld devices to desktop computers. Currently desktop metaphors applied to 
small form factor devices are cluttered and difficult to use. 

• Reduce complexity and confusion of the multi-windowed interface by using a model of a single main 
window. Usability data show that users of multi-windowed systems don't always know which actions 
will produce results in which window. 

• Provide greater flexibility to users, content developers and third-party software developers by 
providing broader capabilities for customizing the user interface and creating dynamic content 

• Provide a save-less model, so that users don't need to explicidy save their work. 

• Provide rich support for audio note taking with the ability to correlate and synchronize audio and 
textual material and to review and retrieve audio notes. 

• Provide automatic and transparent synchronization between a user's computers (e.g. handheld and 
desktop). Current device-to-device and device-to-PC synchronization schemes are not seamless and 
require a great deal of configuration and attention from the user. 

• Make the process of getting help about a function be the same as the process for carrying out the 
function. In today's user interfaces, methods for getting assistance with an interface or with content are 
currently separate from the content and often require completely different interactions than consuming 
the content 

• Make sharing and collaborating on documents easy and automatic. Today's devices and PCs have a 
"single-user model" at the heart of their interface metaphors; sharing content and annotations is 
difficult and non-intuitive. 

Description of the Invention: 

[Describe your proposed implementation of the invention, including the architecture and design details of the implementation. The 
design details should Include a description of the component parts of, and individual operations performed by, your 
implementation. The use of a specific example, showing how the invention solves the problem being addressed, can be particularly 
helpful You should also mention whether you have thought of any other implementations, or applications of your invention. In 
most cases, 1-2 pages of description should be adequate to start the patent application process, although a more detailed 
description may greatly enhance the efficiency of the process.] 

Windowless Interface - The display may be divided into non-windowing regions in which different 
content may be displayed. Each region may be individually navigated without changing focus or * t window" 
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state. Each region may include concurrent active html links without changing focus. The benefits of this 
architecture include reducing the complexity associated with both the design and use of the interface while 
still providing access to and interaction with multiple sources of concurrent content. (See also pinning 
(ePad) 2 Joe.) 

HTML based interface - The interface is implemented entirely as an HTML document, except that 
some links are links to scripts rather than to other documents or sites; activating such a link (such as by 
touching it) runs the linked-to script, thus making the link act as a command. Even the desktop metaphor of 
files and folders is expressed as lists of links on a page. 

Additionally, links have properties that govern their appearance and behavior. For example, a link's 
properties may dictate that it appear as a blue underscored text string as in today's browser's, or as a 3D 
button, as a graphic icon, as a thumbnail image of the content being linked to, or even as an embedded 
frame that is open on the content being linked to. The ability to control a link's appearance and behavioral 
properties makes possible rich authoring and customization of content and the user interface. 

The user may completely customize the interface (unless authored-in content controls prevent editing). 
Because there is no distinction between the interface and the content, the kinds of normal editing commands 
the user might use to alter content can be used to customize the interface. What's more, the many varied 
interface widgets that users currently must understand in order to perform useful work, such as dialog 
boxes, drop-down list boxes, etc can be done away with. Everything is simply a link on a page. Some of 
these links may be rendered as buttons to offer instant-gestalt affordances to new users, but architecturally 
they are links that may be used to search, view, and navigate either the interface or the content being 
viewed. Whether you are moving back/forward, sorting, or bookmarking a content page or an interface 
page, the concept and actions are exactly the same. 

The ability to review files, folders, mail, etc by navigating through a list of the links contemporaneously 
is enabled by this architecture and to a lesser extent by the Windowless UI architecture. History, notes, 
clippings and other system-provided directories have many unique attributes as a result of the HTML-based 
architecture also summarized below. 

Rich Navigation - The interface contains methods for paging within a document (previous/next) and 
for traversing chronology (history through back/forward). A rich model for navigation is enabled by 
attaching properties to links. For example, holding on a link offers additional information about where that 
link will lead (Link look-ahead might be displayed in thumbnail form and further holding might result in a 
graphical map of the links attached to that prospective destination.). Back and forward buttons by default 
display thumbnail views of the pages that tapping that button will lead to. 

History (all the pages the user has viewed) may be viewed in a number of ways: by time, by 
appearance, by site, document, section, page, etc. Since all of history is saved, the users' sequence may be 
viewed in a number of ways: as nodes with side tracking branches, as a linear list, or as a combination of 
most recently viewed pages and last few task categories. 

A key difference between the navigation mechanism of this invention and the prior art is that all 
navigation actions are saved in the navigational history. For example, if a user navigates through a set of 
links then goes **back" several steps, and then links through a different set of links, the user is still able to go 
**back" and then retrace their original path of links they navigated In today's browser's this is not 
possible — recollection of the first set of documents that were ''backed" over is lost. 

Annotations - The interface allows the user to interact with content and interface so as to make it more 
memorable, via bookmarks, clippings, highlights, overlaid and embedded ink and audio notes (See files: 
Book position (ePad) 2.doc, Ink layers on a browser.doc and Bookmark (ePad) 2.doc. as well as additional 
documents to be created.) These annotations may be performed with not only different input methods, but 
adapted to the most convenient or natural one, such as a finger for highlighting, a stylus for writing or 
doodling, and speech for lengthier commentary. 

The annotations exist apart from the content (as files with links to the content); therefore they can be 
displayed not only layered on the content in appropriate positions within the content, but in other locations 
and visualizations. That is, each of these annotations is available to the user not only in situ, in the content 
where it was created, but cross-referenced in automatically-created indices which can be manipulated much 
as described in History above: by time, by appearance, by site, document, section, page, etc. Likewise they 
might be shared, selectively shared or kept private with the kind of functionality described under 
"Collaboration." 
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navigate their view of the shared document to other than the currently shared page, thus automatically 
disengaging from the shared display, and to reengage on the shared page at any time. The interface uses a 
Floor request/grant/relinquish model for controlling the group display. Users are able to request control of 
the current group display, grant the request and relinquish control (if currently in control). Side chats can 
be initiated with other members attending the meeting by using the "hold-for-more-infbrmation" method: 
holding on a visual representation of users in the room and then selecting the chat option from a content 
menu. Since in this interface model, presentations are just documents, users can invoice all the standard 
interface methods such as written and audio note taking, searching, bookmarking, etc. on both the public 
presentation (if they have the floor), or on their individual private copy. Just as with other annotations, these 
methods result in a file stored separately from the content, but linked to it in precisely the locations the user 
annotated. 

Help Book - Because the commands in the interface are simply links on a page, there isn't any 
difference between documentation, help, wizards and the other ways we currently attempt to help a user 
perform an action. Asking for help by clicking on a help link summons an appropriate help page composed 
on the fly from the content of the help book. The principles with underlie this composition are contextual: 
that is, if the user currently in the "list" or '"books" of Mail Messages, the help system infers that requests 
for assistance are about how to perform work with Mail Messages. 

Related Ideas addressed in additional pre-disclosure docs 

1 . Using pinning to preview list of contents, e.g., email, folders, contacts, notes, etc.; navigate within a 
book; or to access different content, (file: Pinning (ePad) 2.doc). Different than Outlook previews 
because it is accomplished through html links and can be done with any list of objects. 

2. History can be manipulated, sorted, viewed in different ways, and the like. One feature of the history 
implementation is maintaining all branch extremities and including flexibility for presenting different 
arrangements for viewing the branches. 

3. Notes can also be manipulated, sorted, viewed in different ways but also can be selected resulting in not 
only opening the note but also the book where the note is linked. 

4. Unique thumbnail view UI for notes, history and other objects. 

5. Commands implemented through links 

Diagrams and Flow Charts: 

[To support the description provided above, please include: (a) at least one block diagram showing the architecture of the system 
that implements your invention, and (b) at least one diagram illustrating the primary steps performed by your invention J 
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Appl. No. 
Applicant 
Filed 
Title 



09/456,249 

Darryl E. Rubin et al. 

December 7, 1999 

Computer User Interface Architecture Wherein Users 
Interact With Both Content And User Interface By 
Activating Links 



TC/A.U. 
Examiner 

Docket No. 
Confirmation No. 



2176 

Almari Romero Yuan 

003797.85089 
6181 



Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 



DECLARATION UNDER 37 C.F.R. S 1,131 



Sir: 



We, DARRYL E. RUBIN, ANDREW C. BAIRD, JOHN L. BEEZER, JONATHAN C 
CLUTS, and SUSAN D. WOOLF, hereby declare 1 that: 



1) We are named as joint inventors of the above-captioned application, U.S. Application 
Serial No. 09/456,249, and all claims presently pending therein; 

2) I, DARRYL E. RUBIN, am presently employed by Microsoft Corporation 
(Microsoft) and have been since June 1986. Microsoft is the assignee of the above- 
identified application. 



1 Each numbered declaration is a joint declaration unless an individual reference has been made. In such a case, the 
referenced individual is making the numbered declaration. 
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Appl. No. 09/456,249 

Declaration under 37 C.F.R. § 1.131 

Reply to Office Action of July 8, 2004 

3) I, JONATHAN C. CLUTS, am presently employed by Microsoft and have been since 
October 1990. Microsoft is the assignee of the above-identified application. 

4) I, SUSAN D. WOOLF, am presently employed by Microsoft and have been since 
October 1995. Microsoft is the assignee of the above-identified application. 

5) We, ANDREW C. BAIRD and JOHN L BEEZER, were formerly employed by 
Microsoft. 

6) We were employed by Microsoft during development of the above-identified 
invention. 

7) Prior to May 8, 1999, the earliest priority date of Bizweb2000.com - screenshots of 
an e-book, published 5/8/1999, pages 1-4 ("Bizweb2000 reference"), we conceived of 
and reduced to practice the invention recited in the claims of the above-captioned 
application. 

8) Conception and actual reduction to practice occurred prior to May 8, 1999, as 
evidenced by pages 1, 2, 4, 5, and 7 of the Patent Predisclosure Document attached as 
Exhibit A. 

9) Support for claims 1-5 of the above-captioned application can be found, among other 
places, at least within Exhibit A prepared prior to May 8, 1999. 

10) Support for claims 1-5 can be found at least within Exhibit A, among other places, in 
the bullet points on page 1, and the descriptions at the bottom of page 1 and the top of 
page 2. In addition, the screenshots on pages 5 and 7 support claims 1-5. Claim 2 is 
additionally supported, among other places, by the description on page 4 of Exhibit 
A. 

11) Attached Exhibit A has not been altered since it was originally prepared, except for 
the redaction of dates and the removal of pages 3, 6, and 8, which are not pertinent to 
claims 1-5. 

12) Each of us individually represents that we are over 18 years of age and of competent 
mind. 
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Declaration under 37 C.F.R. § 1.131 
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13) All statements made of our own knowledge are true and all statements made on 
information and belief are believed to be true; and further, these statements were 
made with the knowledge that willful, false statement so made are punishable by fine 
or imprisonment or both, under 18 U.S.C. § 1001 and that such willful, false 
statements may jeopardize the validity of the above-identified application or any 
patent issuing thereon. 

Respectfully submitted, 



Darryl E. Rubin, Date 
Microsoft Corporation 



Andrew C. Baird Date 





John L. Beezer 



Jonathan C. Cluts 
Microsoft Corporation 



Date 



Susan D. Woolf, 
Microsoft Corporation 



Date 
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Microsoft Patent Predisclosure Document 



Title of Invention: Electronic Web-based Interface for Reading and Annotation 
Date: 

Document Author(s): Darryl Rubin, Jonathan Outs, Suze Woolf 



Introduction 

[Please provide a high level description of the invention, including the names of the people who contributed to the invention,] 

An architecture for a window-less, scroll-less, desktop-less and save-less HTML-based universal viewer 
interface which allows the user to read, annotate, collaborate and perform other tasks typical of knowledge 
work, as well as alter the interface to best suit their work patterns. 

Motivation for the Invention: 

[Describe (I) the problem addressed by the invention (e.g., limitations of prior products of Microsoft, or others), and (2) your 
solution to the problem (including what "new " things your invention does and a high-level description of how it does them).] 

• Create a system that is simpler (fewer commands to achieve comparable results) and more powerful 
(achieve results that weren't possible) than today's desktop metaphor by unifying content and user 
interface into a single, dynamic content mechanism that uses fewer kinds of user interface controls in a 
more general way. 

• Provide a new metaphor for finding, consuming, storing, sharing and organizing content of interest to 
the user because many users find the desktop metaphor confusing and difficult to learn. 

• Provide a framework for more "natural" styles of interacting, employing a minimum number common 
gestures such as touch, hold, erase, draw or write. Currently most desktop computers require a 
keyboard and mouse in order to interact with them; most pen-enabled palmtop computers have 
cumbersome means of interaction. 

• Scales from small handheld devices to desktop computers. Currently desktop metaphors applied to 
small form factor devices are cluttered and difficult to use. 

• Reduce complexity and confusion of the multi-windowed interface by using a model of a single main 
window. Usability data show that users of multi-windowed systems don't always know which actions 
will produce results in which window. 

• Provide greater flexibility to users, content developers and third-party software developers by 
providing broader capabilities for customizing the user interface and creating dynamic content 

• Provide a save-less model, so that users don't need to explicitly save their work. 

• Provide rich support for audio note taking with the ability to correlate and synchronize audio and 
textual material and to review and retrieve audio notes. 

• Provide automatic and transparent synchronization between a user's computers (e.g. handheld and 
desktop). Current device-to-device and device-to-PC synchronization schemes are not seamless and 
require a great deal of configuration and attention from the user. 

• Make the process of getting help about a function be the same as the process for carrying out the 
function. In today's user interfaces, methods for getting assistance with an interface or with content are 
currently separate from the content and often require completely different interactions than consuming 
the content 

• Make sharing and collaborating on documents easy and automatic. Today's devices and PCs have a 
"single-user model" at the heart of their interface metaphors; sharing content and annotations is 
difficult and non-intuitive. 

Description of the Invention: 

[Describe your proposed implementation of the invention, including the architecture and design details of the implementation. The 
design details should Include a description of the component parts of, and individual operations performed by, your 
implementation. The use of a specific example, showing how the invention solves the problem being addressed, can be particularly 
helpful You should also mention whether you have thought of any other implementations, or applications of, your invention. In 
most cases, 7-2 pages of description should be adequate to start the patent application process, although a more detailed 
description may greatly enhance the efficiency of the process.] 

Windowless Interface - The display may be divided into non-windowing regions in which different 
content may be displayed. Each region may be individually navigated without changing focus or "window" 
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state. Each region may include concurrent active html links without changing focus. The benefits of this 
architecture include reducing the complexity associated with both the design and use of the interface while 
still providing access to and interaction with multiple sources of concurrent content. (See also pinning 
(ePad) 2.doc.) 

HTML based interface - The interface is implemented entirely as an HTML document, except that 
some links are links to scripts rather than to other documents or sites; activating such a link (such as by 
touching it) runs the linked-to script, thus making the link act as a command. Even the desktop metaphor of 
files and folders is expressed as lists of links on a page. 

Additionally, links have properties that govern their appearance and behavior. For example, a link's 
properties may dictate that it appear as a blue underscored text string as in today's browser's, or as a 3D 
button, as a graphic icon, as a thumbnail image of the content being linked to, or even as an embedded 
frame that is open on the content being linked to. The ability to control a link's appearance and behavioral 
properties makes possible rich authoring and customization of content and the user interface. 

The user may completely customize the interface (unless authored-in content controls prevent editing). 
Because there is no distinction between the interface and the content, the kinds of normal editing commands 
the user might use to alter content can be used to customize the interface. What's more, the many varied 
interface widgets that users currently must understand in order to perform useful work, such as dialog 
boxes, drop-down list boxes, etc can be done away with. Everything is simply a link on a page. Some of 
these links may be rendered as buttons to offer instant-gestalt affordances to new users, but architecturally 
they are links that may be used to search, view, and navigate either the interface or the content being 
viewed. Whether you are moving back/forward, sorting, or bookmarking a content page or an interface 
page, the concept and actions are exactly the same. 

The ability to review files, folders, mail, etc by navigating through a list of the links contemporaneously 
is enabled by this architecture and to a lesser extent by the Windowless UI architecture. History, notes, 
clippings and other system-provided directories have many unique attributes as a result of the HTML-based 
architecture also summarized below. 

Rich Navigation - The interface contains methods for paging within a document (previous/next) and 
for traversing chronology (history through back/forward). A rich model for navigation is enabled by 
attaching properties to links. For example, holding on a link offers additional information about where that 
link will lead (Link look-ahead might be displayed in thumbnail form and further holding might result in a 
graphical map of the links attached to that prospective destination.). Back and forward buttons by default 
display thumbnail views of the pages that tapping that button will lead to. 

History (all the pages the user has viewed) may be viewed in a number of ways: by time, by 
appearance, by site, document, section, page, etc. Since all of history is saved, the users' sequence may be 
viewed in a number of ways: as nodes with side tracking branches, as a linear list, or as a combination of 
most recently viewed pages and last few task categories. 

A key difference between the navigation mechanism of this invention and the prior art is that all 
navigation actions are saved in the navigational history. For example, if a user navigates through a set of 
links then goes "back" several steps, and then links through a different set of links, the user is still able to go 
"back" and then retrace their original path of links they navigated. In today's browser's this is not 
possible — recollection of the first set of documents that were "backed" over is lost. 

Annotations - The interface allows the user to interact with content and interface so as to make it more 
memorable, via bookmarks, clippings, highlights, overlaid and embedded ink and audio notes (See files: 
Book position (ePad) 2.doc, Ink layers on a browser.doc and Bookmark (ePad) 2.doc. as well as additional 
documents to be created.) These annotations may be performed with not only different input methods, but 
adapted to the most convenient or natural one, such as a finger for highlighting, a stylus for writing or 
doodling, and speech for lengthier commentary. 

The annotations exist apart from the content (as files with links to the content); therefore they can be 
displayed not only layered on the content in appropriate positions within the content, but in other locations 
and visualizations. That is, each of these annotations is available to the user not only in situ, in the content 
where it was created, but cross-referenced in automatically-created indices which can be manipulated much 
as described in History above: by time, by appearance, by site, document, section, page, etc. Likewise they 
might be shared, selectively shared or kept private with the kind of functionality described under 
"Collaboration." 
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navigate their view of the shared document to other than the currently shared page, thus automatically 
disengaging from the shared display, and to reengage on the shared page at any time. The interface uses a 
Floor request/grant/relinquish model for controlling the group display. Users are able to request control of 
the current group display, grant the request and relinquish control (if currently in control). Side chats can 
be initiated with other members attending the meeting by using the "hold-for-more-information" method: 
holding on a visual representation of users in the room and then selecting the chat option from a content 
menu. Since in this interface model, presentations are just documents, users can invoke all the standard 
interface methods such as written and audio note taking, searching, bookmarking, etc. on both the public 
presentation (if they have the floor), or on their individual private copy. Just as with other annotations, these 
methods result in a tile stored separately from the content, but linked to it in precisely the locations the user 
annotated. 

Help Book - Because the commands in the interface are simply links on a page, there isn't any 
difference between documentation, help, wizards and the other ways we currently attempt to help a user 
perform an action. Asking for help by clicking on a help link summons an appropriate help page composed 
on the fly from the content of the help book. The principles with underlie this composition are contextual: 
that is, if the user currently in the "list" or "books" of Mail Messages, the help system infers that requests 
for assistance are about how to perform work with Mail Messages. 

Related Ideas addressed in additional pre-disclosure docs 

1 . Using pinning to preview list of contents, e.g., email, folders, contacts, notes, etc.; navigate within a 
book; or to access different content, (file: Pinning (ePad) 2.doc). Different than Oudook previews 
because it is accomplished through html links and can be done with any list of objects. 

2. History can be manipulated, sorted, viewed in different ways, and the like. One feature of the history 
implementation is maintaining all branch extremities and including flexibility for presenting different 
arrangements for viewing the branches. 

3. Notes can also be manipulated, sorted, viewed in different ways but also can be selected resulting in not 
only opening the note but also the book where the note is linked. 

4. Unique thumbnail view UI for notes, history and other objects. 

5. Commands implemented through links 

Diagrams and Flow Charts: 

[To support the description provided above, please include: (a) at least one block diagram showing the architecture of the system 
that implements your invention, and (b) at least one diagram illustrating the primary steps performed by your invention.] 
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Appl. No. 
Applicant 
Filed 
Title 



09/456,249 

Darryl E. Rubin et al. 

December 7, 1999 

Computer User Interface Architecture Wherein Users 
Interact With Both Content And User Interface By 
Activating Links 



TC/A.U. 
Examiner 

Docket No. 
Confirmation No. 



2176 

Almari Romero Yuan 

003797.85089 
6181 



Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 223 1 3-1 450 



DECLARATION UNDER 37 C.F.R. 8 1.131 



Sir: 



We, DARRYL E. RUBIN, ANDREW C. BAIRD, JOHN L. BEEZER, JONATHAN C 
CLUTS, and SUSAN D. WOOLF, hereby declare 1 that: 



1) We are named as joint inventors of the above-captioned application, U.S. Application 
Serial No. 09/456,249, and all claims presently pending therein; 

2) I, DARRYL E. RUBIN, am presently employed by Microsoft Corporation 
(Microsoft) and have been since June 1986. Microsoft is the assignee of the above- 
identified application. 



Each numbered declaration is a joint declaration unless an individual reference has been made. In such a case the 
referenced individual is making the numbered declaration. 
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Appl. No. 09/456,249 

Declaration under 37 C.F.R. § 1.131 

Reply to Office Action of July 8, 2004 

3) I, JONATHAN C. CLUTS, am presently employed by Microsoft and have been since 
October 1990. Microsoft is the assignee of the above-identified application. 

4) I, SUSAN D. WOOLF, am presently employed by Microsoft and have been since 
October 1995. Microsoft is the assignee of the above-identified application. 

5) We, ANDREW C. BAIRD and JOHN L. BEEZER, were formerly employed by 
Microsoft. 

6) We were employed by Microsoft during development of the above-identified 
invention. 

7) Prior to May 8, 1999, the earliest priority date of Bizweb2000.com - screenshots of 
an e-book, published 5/8/1999, pages 1-4 ("Bizweb2000 reference"), we conceived of 
and reduced to practice the invention recited in the claims of the above-captioned 
application. 

8) Conception and actual reduction to practice occurred prior to May 8, 1999, as 
evidenced by pages 1, 2, 4, 5, and 7 of the Patent Predisclosure Document attached as 
Exhibit A. 

9) Support for claims 1-5 of the above-captioned application can be found, among other 
places, at least within Exhibit A prepared prior to May 8, 1 999. 

10) Support for claims 1-5 can be found at least within Exhibit A, among other places, in 
the bullet points on page 1, and the descriptions at the bottom of page 1 and the top of 
page 2. In addition, the screenshots on pages 5 and 7 support claims 1-5. Claim 2 is 
additionally supported, among other places, by the description on page 4 of Exhibit 
A. 

11) Attached Exhibit A has not been altered since it was originally prepared, except for 
the redaction of dates and the removal of pages 3, 6, and 8, which are not pertinent to 
claims 1-5. 

12) Each of us individually represents that we are over 18 years of age and of competent 
mind. 
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13) All statements made of our own knowledge are true and all statements made on 
information and belief are believed to be true; and further, these statements were 
made with the knowledge that willful, false statement so made are punishable by fine 
or imprisonment or both, under 18 U.S.C § 1001 and that such willful, false 
statements may jeopardize the validity of the above-identified application or any 
patent issuing thereon. 

Respectfully submitted, 



Darryl E. Rubin, Date 
Microsoft Corporation 



Andrew C. Baird Date 



John L. Beezey / Date 





Susan D. Woolf, 
Microsoft Corporation 



Date 
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Microsoft Patent Predisclosure Document 



Title of Invention: Electronic Web-based Interface for Reading and Annotation 
Date: 

Document Author(s): Darryl Rubin, Jonathan Chits, Suze Woolf 



Introduction 

[Please provide a high level description of the invention, including the names of the people who contributed to the invention.] 

An architecture for a window-less, scroll-less, desktop-less and save-less HTML-based universal viewer 
interface which allows the user to read, annotate, collaborate and perform other tasks typical of knowledge 
work, as well as alter the interface to best suit their work patterns. 

Motivation for the Invention: 

[Describe (I) the problem addressed by the invention (e.g., limitations of prior products of Microsoft, or others), and (2) your 
solution to the problem (including what "new" things your invention does and a high-level description of how it does them).] 

• Create a system that is simpler (fewer commands to achieve comparable results) and more powerful 
(achieve results that weren't possible) than today's desktop metaphor by unifying content and user 
interface into a single, dynamic content mechanism that uses fewer kinds of user interface controls in a 
more general way. 

• Provide a new metaphor for finding, consuming, storing, sharing and organizing content of interest to 
the user because many users find the desktop metaphor confusing and difficult to learn. 

• Provide a framework for more "natural" styles of interacting, employing a minimum number common 
gestures such as touch, hold, erase, draw or write. Currently most desktop computers require a 
keyboard and mouse in order to interact with them; most pen-enabled palmtop computers have 
cumbersome means of interaction. 

• Scales from small handheld devices to desktop computers. Currently desktop metaphors applied to 
small form factor devices are cluttered and difficult to use. 

• Reduce complexity and confusion of the multi-windowed interface by using a model of a single main 
window. Usability data show that users of multi-windowed systems don't always know which actions 
will produce results in which window. 

• Provide greater flexibility to users, content developers and third-party software developers by 
providing broader capabilities for customizing the user interface and creating dynamic content. 

• Provide a save-less model, so that users don't need to explicitly save their work. 

• Provide rich support for audio note taking with the ability to correlate and synchronize audio and 
textual material and to review and retrieve audio notes. 

• Provide automatic and transparent synchronization between a user's computers (e.g. handheld and 
desktop). Current device-to-device and device-to-PC synchronization schemes are not seamless and 
require a great deal of configuration and attention from the user. 

• Make the process of getting help about a function be the same as the process for carrying out the 
function. In today's user interfaces, methods for getting assistance with an interface or with content are 
currently separate from the content and often require completely different interactions than consuming 
the content 

• Make sharing and collaborating on documents easy and automatic. Today's devices and PCs have a 
"sing! e-user model" at the heart of their interface metaphors; sharing content and annotations is 
difficult and non-intuitive. 

Description of the Invention: 

[Describe your proposed implementation of the invention, including the architecture and design details of the implementation. The 
design details should Include a description of the component parts of, and individual operations performed by, your 
implementation. The use of a specific example, showing how the invention solves the problem being addressed, can be particularly 
helpful. You should also mention whether you have thought of any other implementations, or applications of, your invention. In 
most cases, 1-2 pages of description should be adequate to start the patent application process, although a more detailed 
description may greatly enhance the efficiency of the process] 

Windowless Interface - The display may be divided into non- windowing regions in which different 
content may be displayed. Each region may be individually navigated without changing focus or "window" 
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state. Each region may include concurrent active html links without changing focus. The benefits of this 
architecture include reducing the complexity associated with both the design and use of the interface while 
still providing access to and interaction with multiple sources of concurrent content. (See also pinning 
(ePad) 2.doc.) 

HTML based interface - The interface is implemented entirely as an HTML document, except that 
some links are links to scripts rather than to other documents or sites; activating such a link (such as by 
touching it) runs the linked-to script, thus making the link act as a command. Even the desktop metaphor of 
files and folders is expressed as lists of links on a page. 

Additionally, links have properties that govern their appearance and behavior. For example, a link's 
properties may dictate that it appear as a blue underscored text string as in today's browser's, or as a 3D 
button, as a graphic icon, as a thumbnail image of the content being linked to, or even as an embedded 
frame that is open on the content being linked to. The ability to control a link's appearance and behavioral 
properties makes possible rich authoring and customization of content and the user interface. 

The user may completely customize the interface (unless authored-in content controls prevent editing). 
Because there is no distinction between the interface and the content, the kinds of normal editing commands 
the user might use to alter content can be used to customize the interface. What's more, the many varied 
interface widgets that users currently must understand in order to perform useful work, such as dialog 
boxes, drop-down list boxes, etc can be done away with. Everything is simply a link on a page. Some of 
these links may be rendered as buttons to offer instant-gestalt affordances to new users, but architecturally 
they are links that may be used to search, view, and navigate either the interface or the content being 
viewed. Whether you are moving back/forward, sorting, or bookmarking a content page or an interface 
page, the concept and actions are exactly the same. 

The ability to review files, folders, mail, etc by navigating through a list of the links contemporaneously 
is enabled by this architecture and to a lesser extent by the Windowless UI architecture. History, notes, 
clippings and other system-provided directories have many unique attributes as a result of the HTML-based 
architecture also summarized below. 

Rich Navigation - The interface contains methods for paging within a document (previous/next) and 
for traversing chronology (history through back/forward). A rich model for navigation is enabled by 
attaching properties to links. For example, holding on a link offers additional information about where that 
link will lead (Link look-ahead might be displayed in thumbnail form and further holding might result in a 
graphical map of the links attached to that prospective destination.). Back and forward buttons by default 
display thumbnail views of the pages that tapping that button will lead to. 

History (all the pages the user has viewed) may be viewed in a number of ways: by time, by 
appearance, by site, document, section, page, etc. Since all of history is saved, the users' sequence may be 
viewed in a number of ways: as nodes with side tracking branches, as a linear list, or as a combination of 
most recently viewed pages and last few task categories. 

A key difference between the navigation mechanism of this invention and the prior art is that all 
navigation actions are saved in the navigational history. For example, if a user navigates through a set of 
links then goes "back" several steps, and then links through a different set of links, the user is still able to go 
"back" and then retrace their original path of links they navigated. In today's browser's this is not 
possible — recollection of the first set of documents that were "backed" over is lost. 

Annotations - The interface allows the user to interact with content and interface so as to make it more 
memorable, via bookmarks, clippings, highlights, overlaid and embedded ink and audio notes (See files: 
Book position (ePad) 2.doc, Ink layers on a browser.doc and Bookmark (ePad) 2.doc. as well as additional 
documents to be created.) These annotations may be performed with not only different input methods, but 
adapted to the most convenient or natural one, such as a finger for highlighting, a stylus for writing or 
doodling, and speech for lengthier commentary. 

The annotations exist apart from the content (as files with links to the content); therefore they can be 
displayed not only layered on the content in appropriate positions within the content, but in other locations 
and visualizations. That is, each of these annotations is available to the user not only in situ, in the content 
where it was created, but cross-referenced in automatically-created indices which can be manipulated much 
as described in History above: by time, by appearance, by site, document, section, page, etc. Likewise they 
might be shared, selectively shared or kept private with the kind of functionality described under 
"Collaboration." 
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navigate their view of the shared document to other than the currently shared page, thus automatically 
disengaging from the shared display, and to reengage on the shared page at any time. The interface uses a 
Floor request/grant/relinquish model for controlling the group display. Users are able to request control of 
the current group display, grant the request and relinquish control (if currently in control). Side chats can 
be initiated with other members attending the meeting by using the "hold-for-more-information" method: 
holding on a visual representation of users in the room and then selecting the chat option from a content 
menu. Since in this interface model, presentations are just documents, users can invoke all the standard 
interface methods such as written and audio note taking, searching, bookmarking, etc. on both the public 
presentation (if they have the floor), or on their individual private copy. Just as with other annotations, these 
methods result in a file stored separately from the content, but linked to it in precisely the locations the user 
annotated. 

Help Book - Because the commands in the interface are simply links on a page, there isn't any 
difference between documentation, help, wizards and the other ways we currently attempt to help a user 
perform an action. Asking for help by clicking on a help link summons an appropriate help page composed 
on the fly from the content of the help book. The principles with underlie this composition are contextual: 
that is, if the user currently in the "list" or "books" of Mail Messages, the help system infers that requests 
for assistance are about how to perform work with Mail Messages. 

Related Ideas addressed in additional pre-disclosure docs 

1 . Using pinning to preview list of contents, e.g., email, folders, contacts, notes, etc.; navigate within a 
book; or to access different content, (file: Pinning (ePad) 2.doc). Different than Outlook previews 
because it is accomplished through html links and can be done with any list of objects. 

2. History can be manipulated, sorted, viewed in different ways, and the like. One feature of the history 
implementation is maintaining all branch extremities and including flexibility for presenting different 
arrangements for viewing the branches. 

3. Notes can also be manipulated, sorted, viewed in different ways but also can be selected resulting in not 
only opening the note but also the book where the note is linked. 

4. Unique thumbnail view UI for notes, history and other objects. 

5. Commands implemented through links 

Diagrams and Flow Charts: 

[To support the description provided above, please include: (a) at least one block diagram showing the architecture of the system 
that implements your invention, and (b) at least one diagram illustrating the primary steps performed by your invention.] 
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DECLARATION UNDER 37 C.F.R. 8 1.131 



Sir: 



We, DARRYL E. RUBIN, ANDREW C. BAIRD, JOHN L. BEEZER, JONATHAN C 
CLUTS, and SUSAN D. WOOLF, hereby declare 1 that: 

1) We are named as joint inventors of the above-captioned application, U.S. Application 
Serial No. 09/456,249, and all claims presently pending therein; 

2) I, DARRYL E. RUBIN, am presently employed by Microsoft Corporation 
(Microsoft) and have been since June 1986. Microsoft is the assignee of the above- 
identified application. 



Each numbered declaration is a joint declaration unless an individual reference has been made. In such a case, the 
referenced individual is making the numbered declaration. 
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Appl. No. 09/456,249 

Declaration under 37 C.F.R. § 1,131 

Reply to Office Action of July 8, 2004 

3) I, JONATHAN C. CLUTS, am presently employed by Microsoft and have been since 
October 1990. Microsoft is the assignee of the above-identified application. 

4) I, SUSAN D. WOOLF, am presently employed by Microsoft and have been since 
October 1995. Microsoft is the assignee of the above-identified application. 

5) We, ANDREW C. BAIRD and JOHN L. BEEZER, were formerly employed by 
Microsoft. 

6) We were employed by Microsoft during development of the above-identified 
invention. 

7) Prior to May 8, 1999, the earliest priority date of Bizweb2000.com - screenshots of 
an e-book, published 5/8/1999, pages 1-4 ("Bizweb2000 reference"), we conceived of 
and reduced to practice the invention recited in the claims of the above-captioned 
application. 

8) Conception and actual reduction to practice occurred prior to May 8, 1999, as 
evidenced by pages 1, 2, 4, 5, and 7 of the Patent Predisclosure Document attached as 
Exhibit A. 

9) Support for claims 1-5 of the above-captioned application can be found, among other 
places, at least within Exhibit A prepared prior to May 8, 1 999. 

10) Support for claims 1-5 can be found at least within Exhibit A, among other places, in 
the bullet points on page 1, and the descriptions at the bottom of page 1 and the top of 
page 2. In addition, the screenshots on pages 5 and 7 support claims 1-5. Claim 2 is 
additionally supported, among other places, by the description on page 4 of Exhibit 
A. 

11) Attached Exhibit A has not been altered since it was originally prepared, except for 
the redaction of dates and the removal of pages 3, 6, and 8, which are not pertinent to 
claims 1-5. 

12) Each of us individually represents that we are over 18 years of age and of competent 
mind. 
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13) All statements made of our own knowledge are true and all statements made on 
information and belief are believed to be true; and further, these statements were 
made with the knowledge that willful, false statement so made are punishable by fine 
or imprisonment or both, under 18 U.S.C. § 1001 and that such willful, false 
statements may jeopardize the validity of the above-identified application or any 
patent issuing thereon. 

Respectfully submitted, 



Darryl E. Rubin, Date 
Microsoft Corporation 



Andrew C. Baird Date 



John L. Beezer Date 



Jonathan C. Cluts Date 
Microsoft Corporation 




Microsoft Corporation 
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Microsoft Patent Predisclosure Document 



Title of Invention: Electronic Web-based Interface for Reading and Annotation 
Date: 

Document Author(s): Darryl Rubin, Jonathan Chits, Suze Woolf 



Introduction 

[Please provide a high level description of the invention, including the names of the people who contributed to the invention.] 

An architecture for a window-less, scroll-less, desktop-less and save-less HTML-based universal viewer 
interface which allows the user to read, annotate, collaborate and perform other tasks typical of knowledge 
work, as well as alter the interface to best suit their work patterns. 

Motivation for the Invention: 

[Describe (I) the problem addressed by the invention (e.g., limitations of prior products of Microsoft, or others), and (2) your 
solution to the problem (including what "new" things your invention does and a high-level description of how it does them).] 

• Create a system that is simpler (fewer commands to achieve comparable results) and more powerful 
(achieve results that weren't possible) than today's desktop metaphor by unifying content and user 
interface into a single, dynamic content mechanism that uses fewer kinds of user interface controls in a 
more general way. 

• Provide a new metaphor for finding, consuming, storing, sharing and organizing content of interest to 
the user because many users find the desktop metaphor confusing and difficult to learn. 

• Provide a framework for more "natural" styles of interacting, employing a minimum number common 
gestures such as touch, hold, erase, draw or write. Currently most desktop computers require a 
keyboard and mouse in order to interact with them; most pen-enabled palmtop computers have 
cumbersome means of interaction. 

• Scales from small handheld devices to desktop computers. Currently desktop metaphors applied to 
small form factor devices are cluttered and difficult to use. 

• Reduce complexity and confusion of the multi-windowed interface by using a model of a single mam 
window. Usability data show that users of multi-windowed systems don't always know which actions 
will produce results in which window. 

• Provide greater flexibility to users, content developers and third-party software developers by 
providing broader capabilities for customizing the user interface and creating dynamic content 

• Provide a save-less model, so that users don't need to explicitly save their work. 

• Provide rich support for audio note taking with the ability to correlate and synchronize audio and 
textual material and to review and retrieve audio notes. 

• Provide automatic and transparent synchronization between a user's computers (e.g. handheld and 
desktop). Current device-to-device and device-to-PC synchronization schemes are not seamless and 
require a great deal of configuration and attention from the user. 

• Make the process of getting help about a function be the same as the process for carrying out the 
function. In today's user interfaces, methods for getting assistance with an interface or with content are 
currently separate from the content and often require completely different interactions than consuming 
the content 

• Make sharing and collaborating on documents easy and automatic. Today's devices and PCs have a 
"single-user model" at the heart of their interface metaphors; sharing content and annotations is 
difficult and non-intuitive. 

Description of the Invention: 

[Describe your proposed implementation of the invention, including the architecture and design details of the implementation. The 
design details should include a description of the component parts of, and individual operations performed by, your 
implementation. The use of a specific example, showing how the invention solves the problem being addressed, can be particularly 
helpful You should also mention whether you have thought of any other implementations, or applications of, your invention. In 
most cases, 1-2 pages of description should be adequate to start the patent application process, although a more detailed 
description may greatly enhance the efficiency of the process.] 

Windowless Interface - The display may be divided into non-windowing regions in which different 
content may be displayed. Each region may be individually navigated without changing focus or "window" 
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state. Each region may include concurrent active html links without changing focus. The benefits of this 
architecture include reducing the complexity associated with both the design and use of the interface while 
still providing access to and interaction with multiple sources of concurrent content. (See also pinning 
(ePad) 2.doc.) 

HTML based interface - The interface is implemented entirely as an HTML document, except that 
some links are links to scripts rather than to other documents or sites; activating such a link (such as by 
touching it) runs the linked-to script, thus making the link act as a command. Even the desktop metaphor of 
files and folders is expressed as lists of links on a page. 

Additionally, links have properties that govern their appearance and behavior. For example, a link's 
properties may dictate that it appear as a blue underscored text string as in today's browser's, or as a 3D 
button, as a graphic icon, as a thumbnail image of the content being linked to, or even as an embedded 
frame that is open on the content being linked to. The ability to control a link's appearance and behavioral 
properties makes possible rich authoring and customization of content and the user interface. 

The user may completely customize the interface (unless authored-in content controls prevent editing). 
Because there is no distinction between the interface and the content, the kinds of normal editing commands 
the user might use to alter content can be used to customize the interface. What's more, the many varied 
interface widgets that users currently must understand in order to perform useful work, such as dialog 
boxes, drop-down list boxes, etc can be done away with. Everything is simply a link on a page. Some of 
these links may be rendered as buttons to offer instant-gestalt affordances to new users, but architecturally 
they are links that may be used to search, view, and navigate either the interface or the content being 
viewed. Whether you are moving back/forward, sorting, or bookmarking a content page or an interface 
page, the concept and actions are exactly the same. 

The ability to review files, folders, mail, etc by navigating through a list of the links contemporaneously 
is enabled by this architecture and to a lesser extent by the Windowless UI architecture. History, notes, 
clippings and other system-provided directories have many unique attributes as a result of the HTML-based 
architecture also summarized below. 

Rich Navigation - The interface contains methods for paging within a document (previous/next) and 
for traversing chronology (history through back/forward). A rich model for navigation is enabled by 
attaching properties to links. For example, holding on a link offers additional information about where that 
link will lead (Link look-ahead might be displayed in thumbnail form and further holding might result in a 
graphical map of the links attached to that prospective destination.). Back and forward buttons by default 
display thumbnail views of the pages that tapping that button will lead to. 

History (all the pages the user has viewed) may be viewed in a number of ways: by time, by 
appearance, by site, document, section, page, etc. Since all of history is saved, the users' sequence may be 
viewed in a number of ways: as nodes with side tracking branches, as a linear list, or as a combination of 
most recently viewed pages and last few task categories. 

A key difference between the navigation mechanism of this invention and the prior art is that all 
navigation actions are saved in the navigational history. For example, if a user navigates through a set of 
links then goes "back" several steps, and then links through a different set of links, the user is still able to go 
"back" and then retrace their original path of links they navigated In today's browser's this is not 
possible — recollection of the first set of documents that were "backed" over is lost 

Annotations - The interface allows the user to interact with content and interface so as to make it more 
memorable, via bookmarks, clippings, highlights, overlaid and embedded ink and audio notes (See files: 
Book position (ePad) 2.doc, Ink layers on a browser Aoc and Bookmark (ePad) 2.doc. as well as additional 
documents to be created.) These annotations may be performed with not only different input methods, but 
adapted to the most convenient or natural one, such as a finger for highlighting, a stylus for writing or 
doodling, and speech for lengthier commentary. 

The annotations exist apart from the content (as files with links to the content); therefore they can be 
displayed not only layered on the content in appropriate positions within the content, but in other locations 
and visualizations. That is, each of these annotations is available to the user not only in situ, in the content 
where it was created, but cross-referenced in automatically-created indices which can be manipulated much 
as described in History above: by time, by appearance, by site, document, section, page, etc. Likewise they 
might be shared, selectively shared or kept private with the kind of functionality described under 
"Collaboration." 
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navigate their view of the shared document to other than the currently shared page, thus automatically 
disengaging from the shared display, and to reengage on the shared page at any time. The interface uses a 
Floor request/grant/relinquish model for controlling the group display. Users are able to request control of 
the current group display, grant the request and relinquish control (if currently in control). Side chats can 
be initiated with other members attending the meeting by using the "hold-for-more-mformation" method: 
holding on a visual representation of users in the room and then selecting the chat option from a content 
menu. Since in this interface model, presentations are just documents, users can invoke all the standard 
interface methods such as written and audio note taking, searching, bookmarking, etc. on both the public 
presentation (if they have the floor), or on their individual private copy. Just as with other annotations, these 
methods result in a file stored separately from the content, but linked to it in precisely the locations the user 
annotated. 

Help Book - Because the commands in the interface are simply links on a page, there isn't any 
difference between documentation, help, wizards and the other ways we currently attempt to help a user 
perform an action. Asking for help by clicking on a help link summons an appropriate help page composed 
on the fly from the content of the help book. The principles with underlie this composition are contextual: 
that is, if the user currently in the "list" or "books" of Mail Messages, the help system infers that requests 
for assistance are about how to perform work with Mail Messages. 

Related Ideas addressed in additional pre-disclosure docs 

1 . Using pinning to preview list of contents, e.g., email, folders, contacts, notes, etc.; navigate within a 
book; or to access different content, (file: Pinning (ePad) 2.doc). Different than Outlook previews 
because it is accomplished through html links and can be done with any list of objects. 

2. History can be manipulated, sorted, viewed in different ways, and the like. One feature of the history 
implementation is maintaining all branch extremities and including flexibility for presenting different 
arrangements for viewing the branches. 

3. Notes can also be manipulated, sorted, viewed in different ways but also can be selected resulting in not 
only opening the note but also the book where the note is linked. 

4. Unique thumbnail view UI for notes, history and other objects. 

5. Commands implemented through links 

Diagrams and Flow Charts: 

[To support the description provided above, please include: (a) at least one block diagram showing the architecture of the system 
that implements your invention, and (b) at least one diagram illustrating the primary steps performed by your invention.] 
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